Our current experiences at UBEP
| Project | Data | Records | Tool | Test |
|---|---|---|---|---|
| Extending SRs | citation set | 7494 (14 SR) |
RF/SVM | 0.934-0.999 (AUC-ROC) |
| VZV detection | Pediatrician free-text notes |
60659 | RNN | 0.953 (AUC-ROC) |
| Otitis Classification | Pediatrician free-text notes |
297673 | RNN | 0.955 (Balance F1) |
| Injuries Classification | ED discharge free-text notes |
8194 | GPT-4 | 99.5-1.0 (Accuracy) |
| SR Screening | citation set | 3080 | Humans + ASReview | 0.96-0.98 (AUC-ROC) |
| SR Screening | citation set | 24931 | GPT-4o | fine-tuning |
| SR Screening | citation set | 535+ | GPT-4o | developing |
| Classification, Extraction & Matching |
citation & registries set | 594587 | GPT-4o | running |
| Model | Type | Access | Computation | Accuracy (n = 320) |
|---|---|---|---|---|
| GPT-4o | pretrained | payed | remote | TBD |
| GPT-3.5-turbo | pretrained | payed | remote | TBD |
| LLama3 70B | pretrained | free | local | 98.6 (97.6 balanced) |
| LLama3 8B | pretrained | free | local | 92.2 (90.8 balanced) |
| Gemma 1.1 7B | pretrained | free | local | 79.1 (75.7 balanced) |
| RF | fitted | self-developed | local | 77.6 |
| GBM | fitted | self-developed | local | 62.8 |
| NB | fitted | self-developed | local | 60.6 |
| SVM | fitted | self-developed | local | 57.8 |
Slides: https://bit.ly/ai4a-text
Artificial Intelligence for Africa
Shaping global health perspective from research and medicine in Africa